This project collected data from the Taylor Swift Lyrics dataset available on Kaggle, which includes the album, song title, and lyrics for each song. For this project, only a selection of 10 albums will be utilized, as they are considered to be Taylor Swift's primary albums.
This section of the code performs preprocessing on the lyrics, which involves removing punctuation, spell-checking the text, and then appending it to a CSV file called "song_lyrics_df_v1.csv" to consolidate the lyrics and song titles.
The Spotify stream counts are obtained from Kworb.net and are manually added to the CSV file. Additionally, the album release years are entered manually and gathered from Music Fandom and The Reading Order.
skip this part and run the next section to load the analysis results to be used for report generation
The song_lyrics_df.csv dataset, which was generated in the previous section of this notebook, is loaded along with the manually inputted data.
In this section, sentiment analysis is performed on the lyrics, and the resulting sentiment scores are exported to a CSV file. Running the following models:
This section imports the sentiment analysis data from the models to be used to generate reports.
song_emotions are 34 randomly picked songs which are tagged and labelled manually as "ground truth".
-------- Accuracy Report -------- DataFrame: DistilBERT Emotion Accuracy: 0.7058823529411765 --------------------------------- DataFrame: AlBERT base Emotion Accuracy: 0.7352941176470589 --------------------------------- DataFrame: Electra base Emotion Accuracy: 0.7647058823529411 ---------------------------------
In the album "1989," there is a notable presence of "joy" alongside a significant emphasis on "sadness." This suggests a range of emotional experiences and reflections within the album. Moving on to "Evermore," we observe a relatively low occurrence of "anger" but a higher focus on "sadness," indicating a somber and introspective tone. In "Fearless," there is a combination of "joy" and "sadness," possibly representing a contrast between uplifting and melancholic moments. "Folklore" stands out with instances of "anger," "fear," and "sadness," suggesting a deeper exploration of complex and darker emotions. The album "Lover" exhibits a mix of emotions, including "fear," "joy," "love," and a notable presence of "sadness," which may reflect a range of romantic experiences. "Midnights" showcases a balance between "joy" and "sadness," possibly conveying a sense of introspection and reflection during nighttime. In "Red," we see a diverse range of emotions with instances of "anger," "fear," "joy," "love," and a significant focus on "sadness," indicating a wide emotional spectrum. "Reputation" leans more towards "sadness" and "joy" while featuring a minor presence of "fear," potentially exploring themes of reputation, vulnerability, and personal growth. Finally, "SpeakNow" is characterized by instances of "anger," "fear," and "sadness," suggesting a mix of intense and emotional experiences. By grouping the data by albums and examining the emotional themes present, we gain insights into the overarching emotional journeys conveyed in each album.
Word cloud visual representation of commonly used words found in lyrics according to their frequency or importance. Words like 'love', 'never', 'cause', 'night', and 'see' among many others are prevalent in Taylor's song writing. These words can build upon themes of emotions that resonate to her fans and audiences.
(-0.5, 499.5, 499.5, -0.5)
Emotion distribution of each albums throughout her collection.
As seen on the graphs below, the 3 models gathered different results from the collated tagged best emotions in each album.
DistilBERT:Joy is one of the leading emotions across the albums followed by sadness.
Electra:Sadness is the leading emotion found across her albums, followed by joy. Surprisingly, love is the lowest tagged emotion through her album.
AlBERTa:Joy is the most common emotion found across her albums, followed by a scattered scores for love, anger and sadness.
The three datasets (alBERT, DistilBERT, and Electra) exhibit variations in emotional distributions across Taylor Swift's albums. In the alBERT dataset, emotions such as anger, joy, and sadness are well-represented, with joy being the most prevalent. The DistilBERT dataset introduces additional emotions such as surprise and fear, while adjusting the counts for certain emotions compared to the alBERT dataset. In this dataset, joy remains prominent, but sadness appears to be more prevalent than anger. The Electra dataset further diverges by emphasizing emotions like sadness and joy, with a significant increase in the count of sadness in some albums. Fear also emerges as a distinct emotion in certain songs. Overall, the datasets demonstrate differences in emotion representation and highlight the variations in emotional themes captured by different models.
The graph below shoes the Spotify Streams by Album with disected emotions. We're displaying results from the AlBERT base Emotion where fear and anger are apparent in the albums with the most streams - 1989, Reputation and Lover. Followed by Folklore, midights and Evermore where sadness plays a big role in the albums.
The analysis reveals interesting insights about Taylor Swift's albums and their associated best emotions and total stream counts. Among her albums, "1989" emerges as the most streamed, with joy and sadness being the predominant emotions. Notably, "Evermore" resonates with listeners through its expression of anger, joy, and sadness, with sadness accumulating the highest total stream count. The album "Fearless" strikes a chord with emotions such as fear, joy, and sadness, with sadness garnering the highest stream count. "Folklore" encompasses a wide range of emotions, including anger, fear, joy, and sadness, with sadness leading in terms of total streams. "Lover" captivates listeners with joy, sadness, and love, while sadness prevails as the most streamed emotion. "Midnights" presents a mix of anger, joy, and sadness, with sadness taking the lead in total streams. In the album "Red," emotions like anger, fear, joy, love, and sadness coexist, with sadness accumulating the highest stream count. The album "Reputation" showcases emotions of fear, joy, and sadness, with sadness amassing the highest total stream count. Lastly, "SpeakNow" elicits emotions of anger, fear, joy, and sadness, with sadness garnering the highest total stream count.